Image super-resolution is a common task on mobile and IoT devices, where one often needs to upscale and enhance low-resolution images and video frames. While numerous solutions have been proposed for this problem in the past, they are usually not compatible with low-power mobile NPUs having many computational and memory constraints. In this Mobile AI challenge, we address this problem and propose the participants to design an efficient quantized image super-resolution solution that can demonstrate a real-time performance on mobile NPUs. The participants were provided with the DIV2K dataset and trained INT8 models to do a high-quality 3X image upscaling. The runtime of all models was evaluated on the Synaptics VS680 Smart Home board with a dedicated edge NPU capable of accelerating quantized neural networks. All proposed solutions are fully compatible with the above NPU, demonstrating an up to 60 FPS rate when reconstructing Full HD resolution images. A detailed description of all models developed in the challenge is provided in this paper.

基于深度学习的单图像超分辨率（SISR）方法引起了人们的关注，并在现代高级GPU上取得了巨大的成功。但是，大多数最先进的方法都需要大量参数，记忆和计算资源，这些参数通常会显示在当前移动设备CPU/NPU上时显示出较低的推理时间。在本文中，我们提出了一个简单的普通卷积网络，该网络具有快速最近的卷积模块（NCNET），该模块对NPU友好，可以实时执行可靠的超级分辨率。提出的最近的卷积具有与最近的UP采样相同的性能，但更快，更适合Android NNAPI。我们的模型可以很容易地在具有8位量化的移动设备上部署，并且与所有主要的移动AI加速器完全兼容。此外，我们对移动设备上的不同张量操作进行了全面的实验，以说明网络体系结构的效率。我们的NCNET在DIV2K 3X数据集上进行了训练和验证，并且与其他有效的SR方法的比较表明，NCNET可以实现高保真SR结果，同时使用更少的推理时间。我们的代码和预估计的模型可在\ url {https://github.com/algolzw/ncnet}上公开获得。

随着LIDAR传感器在自动驾驶中的流行率，3D对象跟踪受到了越来越多的关注。在点云序列中，3D对象跟踪旨在预测给定对象模板中连续帧中对象的位置和方向。在变压器成功的驱动下，我们提出了点跟踪变压器（PTTR），它有效地预测了高质量的3D跟踪，借助变压器操作，以粗到1的方式导致。 PTTR由三个新型设计组成。 1）我们设计的关系意识采样代替随机抽样，以在亚采样过程中保留与给定模板相关的点。 2）我们提出了一个点关系变压器，以进行有效的特征聚合和模板和搜索区域之间的特征匹配。 3）基于粗糙跟踪结果，我们采用了一个新颖的预测改进模块，通过局部特征池获得最终的完善预测。此外，以捕获对象运动的鸟眼视图（BEV）的有利特性（BEV）的良好属性，我们进一步设计了一个名为PTTR ++的更高级的框架，该框架既包含了点的视图和BEV表示）产生高质量跟踪结果的影响。 PTTR ++实质上提高了PTTR顶部的跟踪性能，并具有低计算开销。多个数据集的广泛实验表明，我们提出的方法达到了卓越的3D跟踪准确性和效率。

在本文中，我们提出了与IEEE计算机协会在CVPR 2022上同时与IEEE计算机协会研讨会同时举行的多手术检测挑战。我们的多手术检测挑战旨在检测自动图像操作，包括但不限于图像编辑，图像合成，图像合成，图像，图像，图像，图像合成，图像，图像编辑一代，图像Photoshop等。我们的挑战吸引了来自世界各地的674支团队，约有2000个有效的结果提交数量。我们邀请了前十支球队为挑战提供解决方案，其中三支球队在大结局中获得了奖项。在本文中，我们介绍了前三名团队的解决方案，以增强图像伪造检测领域的研究工作。

由不同形状和非线性形状变化引起的机器官的大变形，对医学图像配准产生了重大挑战。传统的注册方法需要通过特定变形模型迭代地优化目标函数以及细致的参数调谐，但在具有大变形的图像中具有有限的能力。虽然基于深度学习的方法可以从输入图像到它们各自的变形字段中的复杂映射，但它是基于回归的，并且容易被卡在局部最小值，特别是当涉及大变形时。为此，我们呈现随机策划者 - 演员 - 评论家（SPAC），这是一种新的加强学习框架，可以执行逐步登记。关键概念通过每次步骤连续地翘曲运动图像，以最终与固定图像对齐。考虑到在传统的强化学习（RL）框架中处理高维连续动作和状态空间有挑战性，我们向标准演员 - 评论家模型引入了一个新的概念“计划”，这是低维度，可以促进演员生成易于高维行动。整个框架基于无监督的培训，并以端到端的方式运行。我们在几个2D和3D医学图像数据集上评估我们的方法，其中一些包含大变形。我们的经验结果强调了我们的工作实现了一致，显着的收益和优于最先进的方法。

训练无模型的深度加强学习模型来解决图像到图像转换是困难的，因为它涉及高维连续状态和动作空间。在本文中，我们借鉴了最近的最大熵增强学习框架成功的灵感来设计用于挑战连续控制问题，在包括图像表示，产生和控制的高维连续空间上开发随机策略。这种方法的核心是随机演员 - 执行程序 - 批评者 - 评论家（SAEC），这是一个违法的演员 - 评论家模型，具有额外的excator来生成现实图像。具体地，该actor通过随机潜行动作侧重于高级表示和控制策略，以及明确地指示执行器生成用于操纵状态的低级动作。关于若干图像到图像转换任务的实验已经证明了在面对高维连续空间问题时所提出的SAEC的有效性和稳健性。

Facial image manipulation has achieved great progress in recent years. However, previous methods either operate on a predefined set of face attributes or leave users little freedom to interactively manipulate images. To overcome these drawbacks, we propose a novel framework termed MaskGAN, enabling diverse and interactive face manipulation. Our key insight is that semantic masks serve as a suitable intermediate representation for flexible face manipulation with fidelity preservation. MaskGAN has two main components: 1) Dense Mapping Network (DMN) and 2) Editing Behavior Simulated Training (EBST). Specifically, DMN learns style mapping between a free-form user modified mask and a target image, enabling diverse generation results. EBST models the user editing behavior on the source mask, making the overall framework more robust to various manipulated inputs. Specifically, it introduces dual-editing consistency as the auxiliary supervision signal. To facilitate extensive studies, we construct a large-scale high-resolution face dataset with fine-grained mask annotations named CelebAMask-HQ. MaskGAN is comprehensively evaluated on two challenging tasks: attribute transfer and style copy, demonstrating superior performance over other state-of-the-art methods. The code, models, and dataset are available at https://github.com/switchablenorms/CelebAMask-HQ.

Weakly-supervised object localization aims to indicate the category as well as the scope of an object in an image given only the image-level labels. Most of the existing works are based on Class Activation Mapping (CAM) and endeavor to enlarge the discriminative area inside the activation map to perceive the whole object, yet ignore the co-occurrence confounder of the object and context (e.g., fish and water), which makes the model inspection hard to distinguish object boundaries. Besides, the use of CAM also brings a dilemma problem that the classification and localization always suffer from a performance gap and can not reach their highest accuracy simultaneously. In this paper, we propose a casual knowledge distillation method, dubbed KD-CI-CAM, to address these two under-explored issues in one go. More specifically, we tackle the co-occurrence context confounder problem via causal intervention (CI), which explores the causalities among image features, contexts, and categories to eliminate the biased object-context entanglement in the class activation maps. Based on the de-biased object feature, we additionally propose a multi-teacher causal distillation framework to balance the absorption of classification knowledge and localization knowledge during model training. Extensive experiments on several benchmarks demonstrate the effectiveness of KD-CI-CAM in learning clear object boundaries from confounding contexts and addressing the dilemma problem between classification and localization performance.

In this paper, a semantic communication framework for image transmission is developed. In the investigated framework, a set of servers cooperatively transmit images to a set of users utilizing semantic communication techniques. To evaluate the performance of studied semantic communication system, a multimodal metric is proposed to measure the correlation between the extracted semantic information and the original image. To meet the ISS requirement of each user, each server must jointly determine the semantic information to be transmitted and the resource blocks (RBs) used for semantic information transmission. We formulate this problem as an optimization problem aiming to minimize each server's transmission latency while reaching the ISS requirement. To solve this problem, a value decomposition based entropy-maximized multi-agent reinforcement learning (RL) is proposed, which enables servers to coordinate for training and execute RB allocation in a distributed manner to approach to a globally optimal performance with less training iterations. Compared to traditional multi-agent RL, the proposed RL improves the valuable action exploration of servers and the probability of finding a globally optimal RB allocation policy based on local observation. Simulation results show that the proposed algorithm can reduce the transmission delay by up to 16.1% compared to traditional multi-agent RL.

New architecture GPUs like A100 are now equipped with multi-instance GPU (MIG) technology, which allows the GPU to be partitioned into multiple small, isolated instances. This technology provides more flexibility for users to support both deep learning training and inference workloads, but efficiently utilizing it can still be challenging. The vision of this paper is to provide a more comprehensive and practical benchmark study for MIG in order to eliminate the need for tedious manual benchmarking and tuning efforts. To achieve this vision, the paper presents MIGPerf, an open-source tool that streamlines the benchmark study for MIG. Using MIGPerf, the authors conduct a series of experiments, including deep learning training and inference characterization on MIG, GPU sharing characterization, and framework compatibility with MIG. The results of these experiments provide new insights and guidance for users to effectively employ MIG, and lay the foundation for further research on the orchestration of hybrid training and inference workloads on MIGs. The code and results are released on https://github.com/MLSysOps/MIGProfiler. This work is still in progress and more results will be published soon.